Various forms of network storage systems exist today, including network attached storage (NAS), storage area networks (SANs), and others. Network storage systems are commonly used for a variety of purposes, such as backing up critical data, providing multiple users with access to shared data, etc.
A network storage system includes at least one storage server, which is a processing system configured to store and retrieve data on behalf of one or more client processing systems (“clients”). In the context of NAS, a storage server is commonly a file server, which is sometimes called a “filer”. A filer operates on behalf of one or more clients to store and manage shared files. The files may be stored in a storage subsystem that includes one or more arrays of mass storage devices, such as magnetic or optical disks or tapes, by using RAID (Redundant Array of Inexpensive Disks). Hence, the mass storage devices in each array may be organized into one or more separate RAID groups.
In a SAN context, a storage server provides clients with access to stored data at a sub-file level of granularity, such as block-level access, rather than file-level access. Some storage servers are capable of providing clients with both file-level access and block-level access, such as certain Filers made by Network Appliance, Inc. (NetApp®) of Sunnyvale, Calif.
Recently, some storage servers have been designed to have distributed architectures, to facilitate clustering of storage nodes. Clustering facilitates scaling of performance and storage capacity. For example, rather than being implemented in a single box, a storage server may include a separate N-(“network”) module and D-(disk) module, which are contained within separate housings and communicate with each other via some type of switching fabric or other communication medium. Each D-module typically manages a separate set of disks. Storage servers which implement the Data ONTAP® GX operating system from NetApp can have this type of distributed architecture.
The design goals associated with network storage systems typically include the goal that the system should scale gracefully with available resources in three important areas: performance, availability and manageability. As an organization grows, to maintain a high level of productivity under larger workloads, its data infrastructure needs to be scaled up in performance as well as storage capacity while keeping costs low. Existing file systems have at least two limitations that inhibit their scalability under popular update-intensive (write-intensive) workloads and complicate data management: (i) the presence of bottleneck resources, and (ii) the need to do load-spreading and data migration tasks manually. For instance, performance and availability are limited as existing cluster designs statically map either all of a volume's files, metadata or directories to a single node, which becomes a single point of contention and failure. Also, administering a cluster becomes increasingly complex as its size is scaled to meet growing organizational needs. Manually performing common data management tasks, such as redistributing data to exploit additional machines or to work around failed machines, incurs significant system downtime and lost productivity.
Some existing clustered file systems scale a cluster's performance for read-mostly workloads by employing front-end caches that replicate data to offload the primary storage node for read accesses. However, replication degrades rather than improves performance for workloads that update (write) data frequently, such as most enterprise-scale document access and software development. To scale performance under such workloads, some clustered file systems statically partition volumes across multiple cluster nodes to spread load, which inhibits their scalability.
Moreover, organizations typically have significant investment in legacy data storage infrastructure, which cannot be easily replaced with newer clustered storage solutions. Such organizations need a performance-scaling solution that integrates well with existing infrastructure.